Continual Optimal Adaptive Tracking Of Uncertain Nonlinear Continuous-time Systems Using Multilayer Neural Networks

This study provides a lifelong integral reinforcement learning (LIRL)-based optimal tracking scheme for uncertain nonlinear continuous-time (CT) systems using multilayer neural network (MNN). In this LIRL framework, the optimal control policies are generated by using both the critic neural network (NN) weights and single-layer NN identifier. The critic MNN weight tuning is accomplished using an improved singular value decomposition (SVD) of its activation function gradient. The NN identifier, on the other hand, provides the control coefficient matrix for computing the control policies. An online weight velocity attenuation (WVA)-based consolidation scheme is proposed wherein the significance of weights is derived by using Hamilton-Jacobi-Bellman (HJB) error. This WVA term is incorporated in the critic MNN update law to overcome catastrophic forgetting. Lyapunov stability is employed to demonstrate the uniform ultimate boundedness of the overall closed-loop system. Finally, a numerical example of a two-link robotic manipulator supports the theoretical claims

Optimal Tracking Of Nonlinear Discrete-time Systems Using Zero-Sum Game Formulation And Hybrid Learning

This paper presents a novel hybrid learning-based optimal tracking method to address zero-sum game problems for partially uncertain nonlinear discrete-time systems. An augmented system and its associated discounted cost function are defined to address optimal tracking. Three multi-layer neural networks (NNs) are utilized to approximate the optimal control and the worst-case disturbance inputs, and the value function. The critic weights are tuned using the hybrid technique, whose weights are updated once at the sampling instants and in an iterative manner over finite times within the sampling instants. The proposed hybrid technique helps accelerate the convergence of the approximated value functional to its actual value, which makes the optimal policy attain quicker. A two-layer NN-based actor generates the optimal control input, and its weights are adjusted based on control input errors. Moreover, the concurrent learning method is utilized to ease the requirement of persistent excitation. Further, the Lyapunov method investigates the stability of the closed-loop system. Finally, the proposed method is evaluated on a two-link robot arm and demonstrates promising results

Lifelong Learning Control of Nonlinear Systems with Constraints using Multilayer Neural Networks with Application to Mobile Robot Tracking

This Paper Presents a Novel Lifelong Multilayer Neural Network (MNN) Tracking Approach for an Uncertain Nonlinear Continuous-Time Strict Feedback System that is Subject to Time-Varying State Constraints. the Proposed Method Uses a Time-Varying Barrier Function to Accommodate the Constraints Leading to the Development of an Efficient Control Scheme. the Unknown Dynamics Are Approximated using a MNN, with Weights Tuned using a Singular Value Decomposition (SVD)-Based Technique. an Online Lifelong Learning (LL) based Elastic Weight Consolidation (EWC) Scheme is Also Incorporated to Alleviate the Issue of Catastrophic Forgetting. the Stability of the overall Closed-Loop System is Analyzed using Lyapunov Analysis. the Effectiveness of the Proposed Method is Demonstrated by using a Quadratic Cost Function through a Numerical Example of Mobile Robot Control Which Demonstrates a 38% Total Cost Reduction When Compared to the Recent Literature and 6% Cost Reduction is Observed When the Proposed Method with LL is Compared to the Proposed Method Without LL

Continual Learning-Based Optimal Output Tracking of Nonlinear Discrete-Time Systems with Constraints: Application to Safe Cargo Transfer

This Paper Addresses a Novel Lifelong Learning (LL)-Based Optimal Output Tracking Control of Uncertain Non-Linear Affine Discrete-Time Systems (DT) with State Constraints. First, to Deal with Optimal Tracking and Reduce the Steady State Error, a Novel Augmented System, Including Tracking Error and its Integral Value and Desired Trajectory, is Proposed. to Guarantee Safety, an Asymmetric Barrier Function (BF) is Incorporated into the Utility Function to Keep the Tracking Error in a Safe Region. Then, an Adaptive Neural Network (NN) Observer is Employed to Estimate the State Vector and the Control Input Matrix of the Uncertain Nonlinear System. Next, an NN-Based Actor-Critic Framework is Utilized to Estimate the Optimal Control Input and the Value Function by using the Estimated State Vector and Control Coefficient Matrix. to Achieve LL for a Multitask Environment in Order to Avoid the Catastrophic Forgetting Issue, the Exponential Weight Velocity Attenuation (EWVA) Scheme is Integrated into the Critic Update Law. Finally, the Proposed Tracker is Applied to a Safe Cargo/ Crew Transfer from a Large Cargo Ship to a Lighter Surface Effect Ship (SES) in Severe Sea Conditions

Lifelong Learning-Based Multilayer Neural Network Control of Nonlinear Continuous-Time Strict-Feedback Systems

In This Paper, We Investigate Lifelong Learning (LL)-Based Tracking Control for Partially Uncertain Strict Feedback Nonlinear Systems with State Constraints, employing a Singular Value Decomposition (SVD) of the Multilayer Neural Networks (MNNs) Activation Function based Weight Tuning Scheme. the Novel SVD-Based Approach Extends the MNN Weight Tuning to (Formula Presented.) Layers. a Unique Online LL Method, based on Tracking Error, is Integrated into the MNN Weight Update Laws to Counteract Catastrophic Forgetting. to Adeptly Address Constraints for Safety Assurances, Taking into Account the Effects Caused by Disturbances, We Utilize a Time-Varying Barrier Lyapunov Function (TBLF) that Ensures a Uniformly Ultimately Bounded Closed-Loop System. the Effectiveness of the Proposed Safe LL MNN Approach is Demonstrated through a Leader-Follower Formation Scenario Involving Unknown Kinematics and Dynamics. Supporting Simulation Results of Mobile Robot Formation Control Are Provided, Confirming the Theoretical Findings

Rate-Based End-to-End Congestion Control of Multimedia Traffic in Packet Switched Networks

This paper proposes an explicit rate-based end-to-end congestion control mechanism to alleviate congestion of multimedia traffic in packet switched networks such as the Internet. The congestion is controlled by adjusting the transmission rates of the sources in response to the feedback information from destination such as the buffer occupancy, packet arrival rate and service rate at the outgoing link, so that a desired quality of service (QoS) can be met. The QoS is defined in terms of packet loss ratio, transmission delay, power, and network utilization. Comparison studies demonstrate the effectiveness of the proposed scheme over New-Reno TCP (a variant of AIMD: additive increase multiplicative decrease) technique during simulated congestion. Since it is end-to-end, no router support is necessary, the proposed methodology can be readily applied to today\u27s Internet, as well as for real-time video and voice data transfer in unicast networks

Adaptive Critic-Based Neural Network Controller for Uncertain Nonlinear Systems with Unknown Deadzones

A multilayer neural network (NN) controller in discrete-time is designed to deliver a desired tracking performance for a class of nonlinear systems with input deadzones. This multilayer NN controller has an adaptive critic NN architecture with two NNs for compensating the deadzone nonlinearity and a third NN for approximating the dynamics of the nonlinear system. A reinforcement learning scheme in discrete-time is proposed for the adaptive critic NN deadzone compensator, where the learning is performed based on a certain performance measure, which is supplied from a critic. The adaptive generating NN rejects the errors induced by the deadzone whereas a second NN based critic generates a signal, which is used to tune the weights of the action generating NN so that the deadzone compensation scheme becomes adaptive whereas a third multilayer NN simultaneously approximate the nonlinear dynamics of the system. Using the Lyapunov approach, the uniform ultimately boundedness (UUB) of the closed-loop tracking error and weight estimates of action generating NN, critic NN and the third NN are shown by using a novel weight update

Optimal Adaptive Output Regulation of Uncertain Nonlinear Discrete-Time Systems using Lifelong Concurrent Learning

This Paper Addresses Neural Network (NN) based Optimal Adaptive Regulation of Uncertain Nonlinear Discrete-Time Systems in Affine Form using Output Feedback Via Lifelong Concurrent Learning. First, an Adaptive NN Observer is Introduced to Estimate Both the State Vector and Control Coefficient Matrix, and its NN Weights Are Adjusted using Both Output Error and Concurrent Learning Term to Relax the Persistency Excitation (PE) Condition. Next, by Utilizing an Actor-Critic Framework for Estimating the Value Functional and Control Policy, the Critic Network Weights Are Tuned Via Both Temporal Different Error and Concurrent Learning Schemes through a Replay Buffer. the Actor NN Weights Are Tuned using Control Policy Errors. to Attain Lifelong Learning for Performing Effectively during Multiple Tasks, an Elastic Weight Consolidation Term is Added to the Critic NN Weight Tuning Law. the State Estimation, Regulation, and the Weight Estimation Errors of the Observer, Actor and Critic NNs Are Demonstrated to Be Bounded When Performing Tasks by using Lyapunov Analysis. Simulation Results Are Carried Out to Verify the Effectiveness of the Proposed Approach on a Vander Pol Oscillator. Finally, Extension to Optimal Tracking is Given Briefly

Robust Neural Network RISE Observer Based Fault Diagnostics And Prediction

A novel fault diagnostics and prediction scheme in continuous time is introduced for a class of nonlinear systems. The proposed method uses a novel neural network (NN) based robust integral sign of the error (RISE) observer, or estimator, allowing for semi-global asymptotic stability in the presence of NN approximation errors, disturbances and unmodeled dynamics. This is in comparison to typical results presented in the literature that show only boundedness in the presence of uncertainties. The output of the observer/estimator is compared with that of the nonlinear system and a residual is used for declaring the presence of a fault when the residual exceeds a user defined threshold. The NN weights are tuned online with no offline tuning phase. The output of the RISE observer is utilized for diagnostics. Additionally, a method for time-to-failure (TTF) prediction, a first step in prognostics, is developed by projecting the developed parameter-update law under the assumption that the nonlinear system satisfies a linear-in-the-parameters (LIP) assumption. The TTF method uses known critical values of a system to predict when an estimated parameter will reach a known failure threshold. The performance of the NN/RISE observer system is evaluated on a nonlinear system and a simply supported beam finite element analysis (FEA) simulation based on laboratory experiments. Results show that the proposed method provides as much as 25% increased accuracy while the TTF scheme renders a more accurate prediction. © 2010 IEEE

Ultra-Lightweight Mutual Authentication Protocol with Substitution Operation

A method comprises using at least one of a reader and a passive tag to exchange values according to an ultra-lightweight mutual authentication protocol with a substitution operation to change a Hamming weight of the values